Using Label Propagation for Learning Temporally Abstract Actions in Reinforcement Learning

Temporal abstraction plays a key role in scaling up reinforcement learning algorithms. While learning and planning with given temporally extended actions has been well studied, the topic of how to construct this type of abstraction automatically from data is still open. We propose to use the label propagation algorithm for community detection in order to construct extended actions, within the framework of options. We illustrate the benefit of the approach in small computational experiments and discuss its relationship to existing methods for subgoal discovery.